A computationally efficient nonparametric sampling (NPS) method of time to event for individual-level models.

Purpose: Individual-level simulation models often require sampling times to events, however efficient parametric distributions for many processes may often not exist. For example, time to death from life tables cannot be accurately sampled from existing parametric distributions. We propose an efficient nonparametric method to sample times to events that does not require any parametric assumption on the hazards. Methods: We developed a nonparametric sampling (NPS) approach that simultaneously draws multiple time-to-event samples from a categorical distribution. This approach can be applied to univariate and multivariate processes. The probabilities for each time interval are derived from the time interval-specific constant hazards. The times to events can then be used directly in individual-level simulation models. We compared the accuracy of our approach in sampling time-to-events from common parametric distributions, including exponential, Gamma, and Gompertz. In addition, we evaluated the method's performance in sampling age to death from US life tables and sampling times to events from parametric baseline hazards with time-dependent covariates. Results: The NPS method estimated similar expected times to events from 1 million draws for the three parametric distributions, 100,000 draws for the homogenous cohort, 200,000 draws from the heterogeneous cohort, and 1 million draws for the parametric distributions with time-varying covariates, all in less than a second. Conclusion: Our method produces accurate and computationally efficient samples for time-to-events from hazards without requiring parametric assumptions. This approach can substantially reduce the computation time required to simulate individual-level models.

Approaches to developing de novo cancer population models to examine questions about cancer and race in bladder, gastric, and endometrial cancer and multiple myeloma: the Cancer Intervention and Surveillance Modeling Network incubator program.

BACKGROUND: We are developing 10 de novo population-level mathematical models in 4 malignancies (multiple myeloma and bladder, gastric, and uterine cancers). Each of these sites has documented disparities in outcome that are believed to be downstream effects of systemic racism. METHODS: Ten models are being independently developed as part of the Cancer Intervention and Surveillance Modeling Network incubator program. These models simulate trends in cancer incidence, early diagnosis, treatment, and mortality for the general population and are stratified by racial subgroup. Model inputs are based on large population datasets, clinical trials, and observational studies. Some core parameters are shared, and other parameters are model specific. All models are microsimulation models that use self-reported race to stratify model inputs. They can simulate the distribution of relevant risk factors (eg, smoking, obesity) and insurance status (for multiple myeloma and uterine cancer) in US birth cohorts and population. DISCUSSION: The models aim to refine approaches in prevention, detection, and management of 4 cancers given uncertainties and constraints. They will help explore whether the observed racial disparities are explainable by inequities, assess the effects of existing and potential cancer prevention and control policies on health equity and disparities, and identify policies that balance efficiency and fairness in decreasing cancer mortality.

A Tutorial on Time-Dependent Cohort State-Transition Models in R Using a Cost-Effectiveness Analysis Example.

In an introductory tutorial, we illustrated building cohort state-transition models (cSTMs) in R, where the state transition probabilities were constant over time. However, in practice, many cSTMs require transitions, rewards, or both to vary over time (time dependent). This tutorial illustrates adding 2 types of time dependence using a previously published cost-effectiveness analysis of multiple strategies as an example. The first is simulation-time dependence, which allows for the transition probabilities to vary as a function of time as measured since the start of the simulation (e.g., varying probability of death as the cohort ages). The second is state-residence time dependence, allowing for history by tracking the time spent in any particular health state using tunnel states. We use these time-dependent cSTMs to conduct cost-effectiveness and probabilistic sensitivity analyses. We also obtain various epidemiological outcomes of interest from the outputs generated from the cSTM, such as survival probability and disease prevalence, often used for model calibration and validation. We present the mathematical notation first, followed by the R code to execute the calculations. The full R code is provided in a public code repository for broader implementation.

An Introductory Tutorial on Cohort State-Transition Models in R Using a Cost-Effectiveness Analysis Example.

Decision models can combine information from different sources to simulate the long-term consequences of alternative strategies in the presence of uncertainty. A cohort state-transition model (cSTM) is a decision model commonly used in medical decision making to simulate the transitions of a hypothetical cohort among various health states over time. This tutorial focuses on time-independent cSTM, in which transition probabilities among health states remain constant over time. We implement time-independent cSTM in R, an open-source mathematical and statistical programming language. We illustrate time-independent cSTMs using a previously published decision model, calculate costs and effectiveness outcomes, and conduct a cost-effectiveness analysis of multiple strategies, including a probabilistic sensitivity analysis. We provide open-source code in R to facilitate wider adoption. In a second, more advanced tutorial, we illustrate time-dependent cSTMs.

Estimating the Impact of Low Influenza Activity in 2020 on Population Immunity and Future Influenza Seasons in the United States.

BACKGROUND: Influenza activity in the 2020-2021 season was remarkably low, likely due to implementation of public health preventive measures such as social distancing, mask wearing, and school closure. With waning immunity, the impact of low influenza activity in the 2020-2021 season on the following season is unknown. METHODS: We built a multistrain compartmental model that captures immunity over multiple influenza seasons in the United States. Compared with the counterfactual case, where influenza activity remained at the normal level in 2020-2021, we estimated the change in the number of hospitalizations when the transmission rate was decreased by 20% in 2020-2021. We varied the level of vaccine uptake and effectiveness in 2021-2022. We measured the change in population immunity over time by varying the number of seasons with lowered influenza activity. RESULTS: With the lowered influenza activity in 2020-2021, the model estimated 102 000 (95% CI, 57 000-152 000) additional hospitalizations in 2021-2022, without changes in vaccine uptake and effectiveness. The estimated changes in hospitalizations varied depending on the level of vaccine uptake and effectiveness in the following year. Achieving a 50% increase in vaccine coverage was necessary to avert the expected increase in hospitalization in the next influenza season. If the low influenza activity were to continue over several seasons, population immunity would remain low during those seasons, with 48% of the population susceptible to influenza infection. CONCLUSIONS: Our study projected a large compensatory influenza season in 2021-2022 due to a light season in 2020-2021. However, higher influenza vaccine uptake would reduce this projected increase in influenza.

BayCANN: Streamlining Bayesian Calibration With Artificial Neural Network Metamodeling.

Purpose: Bayesian calibration is generally superior to standard direct-search algorithms in that it estimates the full joint posterior distribution of the calibrated parameters. However, there are many barriers to using Bayesian calibration in health decision sciences stemming from the need to program complex models in probabilistic programming languages and the associated computational burden of applying Bayesian calibration. In this paper, we propose to use artificial neural networks (ANN) as one practical solution to these challenges. Methods: Bayesian Calibration using Artificial Neural Networks (BayCANN) involves (1) training an ANN metamodel on a sample of model inputs and outputs, and (2) then calibrating the trained ANN metamodel instead of the full model in a probabilistic programming language to obtain the posterior joint distribution of the calibrated parameters. We illustrate BayCANN using a colorectal cancer natural history model. We conduct a confirmatory simulation analysis by first obtaining parameter estimates from the literature and then using them to generate adenoma prevalence and cancer incidence targets. We compare the performance of BayCANN in recovering these "true" parameter values against performing a Bayesian calibration directly on the simulation model using an incremental mixture importance sampling (IMIS) algorithm. Results: We were able to apply BayCANN using only a dataset of the model inputs and outputs and minor modification of BayCANN's code. In this example, BayCANN was slightly more accurate in recovering the true posterior parameter estimates compared to IMIS. Obtaining the dataset of samples, and running BayCANN took 15 min compared to the IMIS which took 80 min. In applications involving computationally more expensive simulations (e.g., microsimulations), BayCANN may offer higher relative speed gains. Conclusions: BayCANN only uses a dataset of model inputs and outputs to obtain the calibrated joint parameter distributions. Thus, it can be adapted to models of various levels of complexity with minor or no change to its structure. In addition, BayCANN's efficiency can be especially useful in computationally expensive models. To facilitate BayCANN's wider adoption, we provide BayCANN's open-source implementation in R and Stan.

Is further research on adult pneumococcal vaccine uptake improvement programs worthwhile? Α value of information analysis.

BACKGROUND: Pneumococcal vaccination policy for US adults is evolving, but previous research has shown that programs to increase vaccine uptake are economically favorable, despite parameter uncertainty. Using value of information (VOI) analysis and prior analyses, we examine the value of further research on vaccine uptake program parameters. METHODS: In US 50-64-year-olds, current vaccine recommendations with and without an uptake program were analyzed. In older adults, current recommendations and an alternative strategy (polysaccharide vaccine for all, adding conjugate vaccine only for the immunocompromised) with and without uptake programs were examined. Uptake program parameters were derived from a clinical trial (absolute uptake increase 12.3% [range 0-25%], per-person cost $1.78 [range $0.70-$2.26]), with other parameters obtained from US databases. VOI analyses incorporated probabilistic sensitivity analysis outputs into R-based regression techniques. RESULTS: In 50-64-year-olds, an uptake program cost $54,900/QALY gained compared to no uptake program. For ages ≥65, the program cost $287,000/QALY gained with the alternative strategy and $765,000/QALY with current recommendations. In younger adults, population-level expected value of perfect information (EVPI) was $59.7 million at $50,000/QALY gained and $2.8 million at $100,000/QALY gained. In older adults, EVPI values ranged from ~$1 million to $34.5 million at $100,000 and $200,000/QALY thresholds. The population expected value of partial perfect information (EVPPI) for combined uptake program cost and uptake improvement parameters in the younger population was $368,700 at $50,000/QALY and $43,900 at $100,000/QALY gained thresholds. In older adults, population EVPPI for vaccine uptake program parameters was $0 at both thresholds, reaching a maximum value of $445,000 at a $225,000/QALY threshold. Other model parameters comprised larger components of the global EVPI. CONCLUSION: VOI results do not support further research on pneumococcal vaccine uptake programs in adults at commonly cited US cost-effectiveness benchmarks. Further research to reduce uncertainty in other aspects of adult pneumococcal vaccination is justifiable.

Future projection of the health and functional status of older people in Japan: A multistate transition microsimulation model with repeated cross-sectional data.

Accurate future projections of population health are imperative to plan for the future healthcare needs of a rapidly aging population. Multistate-transition microsimulation models, such as the U.S. Future Elderly Model, address this need but require high-quality panel data for calibration. We develop an alternative method that relaxes this data requirement, using repeated cross-sectional representative surveys to estimate multistate-transition contingency tables applied to Japan's population. We calculate the birth cohort sex-specific prevalence of comorbidities using five waves of the governmental health surveys. Combining estimated comorbidity prevalence with death record information, we determine the transition probabilities of health statuses. We then construct a virtual Japanese population aged 60 and older as of 2013 and perform a microsimulation to project disease distributions to 2046. Our estimates replicate governmental projections of population pyramids and match the actual prevalence trends of comorbidities and the disease incidence rates reported in epidemiological studies in the past decade. Our future projections of cardiovascular diseases indicate lower prevalence than expected from static models, reflecting recent declining trends in disease incidence and fatality.

Carfentanil and the rise and fall of overdose deaths in the United States.

BACKGROUND AND AIMS: It is widely believed that the 2018 decline in overdose deaths in the United States was attributable to a range of public health interventions, however, this decline also coincided with the regulation and decline in use of potent fentanyl analogs, especially carfentanil. The aim of this study was to investigate the association between overdose deaths and carfentanil availability in the United States. DESIGN: Secondary analysis of drug overdose deaths from the Center for Disease Control and Prevention (CDC) and carfentanil exhibit data from drug seizures submitted to drug crime labs and published by the Drug Enforcement Administration (DEA). Trends in overdose deaths were compared in states with high carfentanil exhibits with states with low or no carfentanil exhibits. SETTING: United States. PARTICIPANTS: A total of 1 035 923 drug overdose death records in the United States from 1979 through 2019 were studied. MEASUREMENTS: The outcomes studied were number of overdose deaths and mortality rates by state. FINDINGS: Drug overdose deaths have been closely tracked along an exponential curve. The years 2016 and 2017 witnessed a hyper-exponential surge with increases in overdose deaths of 11 228 (+21.4%) and 6605 (+10.4%), respectively. Subsequently in 2018, drug overdose deaths declined by -2870 (-4.1%). This rise and then fall coincided with a surge and then decline in carfentanil drug seizure exhibits during these same years: 0 (2015), 1292 (2016), 5857 (2017) and 804 (2018). The majority of carfentanil exhibits were localized to a few states. The 2018 decline in overdose deaths in the top five states with the greatest spike in carfentanil exhibits in 2017 (Ohio, Florida, Pennsylvania, Kentucky and Michigan) was 2848, which accounted for nearly all of the total US decline. CONCLUSIONS: The 2016-2017 acceleration and then 2018 decline in drug overdose deaths in the United States was associated with the sudden rise and then fall of carfentanil availability. Given the regional variation, carfentanil's decreased availability may have contributed to the reduction in overdose deaths in 2018.

Microsimulation Model to Compare Enteral and Parenteral Iron Supplementation in Children With Intestinal Failure.

BACKGROUND: Children with chronic intestinal failure have a high prevalence of anemia, commonly from iron deficiency, leading to frequent blood transfusions. No current guideline exists for iron supplementation in these children. In this analysis, we evaluate the effectiveness and the cost-effectiveness of using parenteral, enteral, and no iron supplementation to reduce blood transfusions. METHODS: We created a microsimulation model of pediatric intestinal failure over a 1-year time horizon. Model outcomes included cost (US dollars), blood transfusions received, and hemoglobin trend. Strategies tested included no supplementation, daily enteral supplementation, and monthly parenteral supplementation. We estimated parameters for the model using an institutional cohort of 55 patients. Model parameters updated each 1-month cycle using 2 regressions. A multivariate mixed-effects linear regression estimated hemoglobin values at the next month based on data from the prior month. A mixed-effects logistic regression on hemoglobin predicted the probability of receiving a blood transfusion in a given month. RESULTS: Compared with no supplementation, both enteral and parenteral iron supplementation reduced blood transfusions required per patient by 0.3 and 0.5 transfusions per year, respectively. Enteral iron cost $34 per avoided blood transfusion. Parenteral iron cost an additional $6600 per avoided blood transfusion compared with enteral iron. CONCLUSIONS: We found both parenteral and enteral iron to be effective at reducing blood transfusions. The cost of enteral iron makes it the desired choice in patients who can tolerate it. Future work should aim to identify which subpopulations of patients may benefit most from one strategy over the other.

Data Needs in Opioid Systems Modeling: Challenges and Future Directions.

INTRODUCTION: The opioid crisis is a pervasive public health threat in the U.S. Simulation modeling approaches that integrate a systems perspective are used to understand the complexity of this crisis and analyze what policy interventions can best address it. However, limitations in currently available data sources can hamper the quantification of these models. METHODS: To understand and discuss data needs and challenges for opioid systems modeling, a meeting of federal partners, modeling teams, and data experts was held at the U.S. Food and Drug Administration in April 2019. This paper synthesizes the meeting discussions and interprets them in the context of ongoing simulation modeling work. RESULTS: The current landscape of national-level quantitative data sources of potential use in opioid systems modeling is identified, and significant issues within data sources are discussed. Major recommendations on how to improve data sources are to: maintain close collaboration among modeling teams, enhance data collection to better fit modeling needs, focus on bridging the most crucial information gaps, engage in direct and regular interaction between modelers and data experts, and gain a clearer definition of policymakers' research questions and policy goals. CONCLUSIONS: This article provides an important step in identifying and discussing data challenges in opioid research generally and opioid systems modeling specifically. It also identifies opportunities for systems modelers and government agencies to improve opioid systems models.

Prioritizing Additional Data Collection to Reduce Decision Uncertainty in the HIV/AIDS Response in 6 US Cities: A Value of Information Analysis.

OBJECTIVES: The ambitious goals of the US Ending the HIV Epidemic initiative will require a targeted, context-specific public health response. Model-based economic evaluation provides useful guidance for decision making while characterizing decision uncertainty. We aim to quantify the value of eliminating uncertainty about different parameters in selecting combination implementation strategies to reduce the public health burden of HIV/AIDS in 6 US cities and identify future data collection priorities. METHODS: We used a dynamic compartmental HIV transmission model developed for 6 US cities to evaluate the cost-effectiveness of a range of combination implementation strategies. Using a metamodeling approach with nonparametric and deep learning methods, we calculated the expected value of perfect information, representing the maximum value of further research to eliminate decision uncertainty, and the expected value of partial perfect information for key groups of parameters that would be collected together in practice. RESULTS: The population expected value of perfect information ranged from $59 683 (Miami) to $54 108 679 (Los Angeles). The rank ordering of expected value of partial perfect information on key groups of parameters were largely consistent across cities and highest for parameters pertaining to HIV risk behaviors, probability of HIV transmission, health service engagement, HIV-related mortality, health utility weights, and healthcare costs. Los Angeles was an exception, where parameters on retention in pre-exposure prophylaxis ranked highest in contributing to decision uncertainty. CONCLUSIONS: Funding additional data collection on HIV/AIDS may be warranted in Baltimore, Los Angeles, and New York City. Value of information analysis should be embedded into decision-making processes on funding future research and public health intervention.

Developing and Validating Metamodels of a Microsimulation Model of Infant HIV Testing and Screening Strategies Used in a Decision Support Tool for Health Policy Makers.

Background. Metamodels can simplify complex health policy models and yield instantaneous results to inform policy decisions. We investigated the predictive validity of linear regression metamodels used to support a real-time decision-making tool that compares infant HIV testing/screening strategies. Methods. We developed linear regression metamodels of the Cost-Effectiveness of Preventing AIDS Complications Pediatric (CEPAC-P) microsimulation model used to predict life expectancy and lifetime HIV-related costs/person of two infant HIV testing/screening programs in South Africa. Metamodel performance was assessed with cross-validation and Bland-Altman plots, showing between-method differences in predicted outcomes against their means. Predictive validity was determined by the percentage of simulations in which the metamodels accurately predicted the strategy with the greatest net health benefit (NHB) as projected by the CEPAC-P model. We introduced a zone of indifference and investigated the width needed to produce between-method agreement in 95% of the simulations. We also calculated NHB losses from "wrong" decisions by the metamodel. Results. In cross-validation, linear regression metamodels accurately approximated CEPAC-P-projected outcomes. For life expectancy, Bland-Altman plots showed good agreement between CEPAC-P and the metamodel (within 1.1 life-months difference). For costs, 95% of between-method differences were within $65/person. The metamodels predicted the same optimal strategy as the CEPAC-P model in 87.7% of simulations, increasing to 95% with a zone of indifference of 0.24 life-months ( ∼ 7 days). The losses in health benefits due to "wrong" choices by the metamodel were modest (range: 0.0002-1.1 life-months). Conclusions. For this policy question, linear regression metamodels offered sufficient predictive validity for the optimal testing strategy as compared with the CEPAC-P model. Metamodels can simulate different scenarios in real time, based on sets of input parameters that can be depicted in a widely accessible decision-support tool.

Computing the Expected Value of Sample Information Efficiently: Practical Guidance and Recommendations for Four Model-Based Methods.

Value of information (VOI) analyses can help policy makers make informed decisions about whether to conduct and how to design future studies. Historically a computationally expensive method to compute the expected value of sample information (EVSI) restricted the use of VOI to simple decision models and study designs. Recently, 4 EVSI approximation methods have made such analyses more feasible and accessible. Members of the Collaborative Network for Value of Information (ConVOI) compared the inputs, the analyst's expertise and skills, and the software required for the 4 recently developed EVSI approximation methods. Our report provides practical guidance and recommendations to help inform the choice between the 4 efficient EVSI estimation methods. More specifically, this report provides: (1) a step-by-step guide to the methods' use, (2) the expertise and skills required to implement the methods, and (3) method recommendations based on the features of decision-analytic problems.

Age and generational patterns of overdose death risk from opioids and other drugs.

The ongoing substance misuse epidemic in the United States is complex and dynamic and should be approached as such in the development and evaluation of policy1. Drug overdose deaths (largely attributable to opioid misuse) in the United States have grown exponentially for almost four decades, but the mechanisms of this growth are poorly understood2. From analysis of 661,565 overdose deaths from 1999 to 2017, we show that the age-specific drug overdose mortality curve for each birth-year cohort rises and falls according to a Gaussian-shaped curve. The ascending portion of each successive birth-year cohort mortality curve is accelerated compared with that of all preceding birth-year cohorts. This acceleration can be attributed to either of two distinct processes: a stable peak age, with an increasing amplitude of mortality rate curves from one birth-year cohort to the next; or a youthward shift in the peak age of the mortality rate curves. The overdose epidemic emerged and increased in amplitude among the 1945-1964 cohort (Baby Boomers), shifted youthward among the 1965-1980 cohort (Generation X), and then resumed the pattern of increasing amplitude in the 1981-1990 Millennials. These shifting age and generational patterns are likely to be driven by socioeconomic factors and drug availability, the understanding of which is important for the development of effective overdose prevention measures.

Calculating the Expected Value of Sample Information in Practice: Considerations from 3 Case Studies.

Background. Investing efficiently in future research to improve policy decisions is an important goal. Expected value of sample information (EVSI) can be used to select the specific design and sample size of a proposed study by assessing the benefit of a range of different studies. Estimating EVSI with the standard nested Monte Carlo algorithm has a notoriously high computational burden, especially when using a complex decision model or when optimizing over study sample sizes and designs. Recently, several more efficient EVSI approximation methods have been developed. However, these approximation methods have not been compared, and therefore their comparative performance across different examples has not been explored. Methods. We compared 4 EVSI methods using 3 previously published health economic models. The examples were chosen to represent a range of real-world contexts, including situations with multiple study outcomes, missing data, and data from an observational rather than a randomized study. The computational speed and accuracy of each method were compared. Results. In each example, the approximation methods took minutes or hours to achieve reasonably accurate EVSI estimates, whereas the traditional Monte Carlo method took weeks. Specific methods are particularly suited to problems where we wish to compare multiple proposed sample sizes, when the proposed sample size is large, or when the health economic model is computationally expensive. Conclusions. As all the evaluated methods gave estimates similar to those given by traditional Monte Carlo, we suggest that EVSI can now be efficiently computed with confidence in realistic examples. No systematically superior EVSI computation method exists as the properties of the different methods depend on the underlying health economic model, data generation process, and user expertise.

Computing PROPr Utility Scores for PROMIS® Profile Instruments.

OBJECTIVES: The Patient-Reported Outcomes Measurement Information System® (PROMIS) Profile instruments measure health status on 8 PROMIS domains. The PROMIS-Preference (PROPr) score provides a preference-based summary score for health states defined by 7 PROMIS domains. The Profile and PROPr share 6 domains; PROPr has 1 unique domain (Cognitive Function-Abilities), and the Profile has 2 unique domains (Anxiety and Pain Intensity). We produce an equation for calculating PROPr utility scores with Profile data. METHODS: We used data from 3982 members of US online survey panels who have scores on all 9 PROMIS domains. We used a 70%/30% split for model fit/validation. Using root-mean-square error and mean error on the utility scale, we compared models for predicting the missing Cognitive Function score via (A) the population average; (B) a score representing excellent cognitive function; (C) a score representing poor cognitive function; (D) a score predicted from linear regression of the 8 profile domains; and (E) a score predicted from a Bayesian neural network of the 8 profile domains. RESULTS: The mean errors in the validation sample on the PROPr scale (which ranges from -0.022 to 1.00) for the models were: (A) 0.025, (B) 0.067, (C) -0.23, (D) 0.018, and (E) 0.018. The root-mean-square errors were: (A) 0.097, (B) 0.12, (C) 0.29, (D) 0.095, and (E) 0.094. CONCLUSION: Although the Bayesian neural network had the best root-mean-square error for producing PROPr utility scores from Profile instruments, linear regression performs almost as well and is easier to use. We recommend the linear model for producing PROPr utility scores for PROMIS Profiles.